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ABSTRACT 

Research and evaluation activities have aradually 
evolved frc .ne selection of an improvement and evaluation model, as 
well as data collection using an attitudinal survey and application 
of case study procedures along with focus group interviews, to the 
preparation of summary documents and specific policy recommendations. 
This document presents models of evaluation, assessment, and 
profiling that have emerged from the aci .vities and findings of the 
Career Ladders Research and Evaluation Project. The project, in its 
5th year of indepth study of 15 pilot-test school districts, has 
produced extensive data on specific interrelated components withm 
school sy:tems* The three resultant assessment models and profiling 
procedures can be applied to any school system (or organization) that 
has identified the essential interrelated components for reaching 
functional goals. The models — "Interrelated Components of Program 
Support and Focus for Effecting Change and Reform in Education," 
"Assessment Model for Projected District Readiness Levels," and 
"District Readiness Profile of Strengths and Weaknessess" — provide 
for individual diversities of readiness levels and allow safficient 
time for change within unique syster.s. (7 references) (KM) 
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FVAT.IJATINO & PROFII ING SCHOOLS 

PASED ON A MODEl. OF ORGANiyiATIO N AL EFFEC TyVENESg; 
Professors & Practioners Collg orating on a Reform Movement 
^o Improve Student Achievement 

Purpose 

The purpose of this report is to present models of evaluation, assessment and 
profiling which have emerged from the activities and findings of the Career Ladders 
Research and Evaluation Project. The evaluation project is in its fifth year of in-depth 
study of 15 pilot-test school distnct organizations. It has produced extensive infomiation 
about specific interrelated components within school systems which must be operating on 
an adequate level in order for change or reform to be effective. 

After summarizing the "research activity" involved in the formulation of the models, 
the researchers v/ill review the follovang three key areas for program evaluation and 
i.iiprovement: (1) assessing organizational readiness levels for program implementation, 
(2) need for adjusting "time and planning for change" to individual districts' operational 
levels; and (3) establishing valid relationships between levels of teacher performance and 
student achievement. 
Rese2ri;h Activity 

Research and evaluation activities have gradually evolved from selection of an 
improvement and evaluation model, as well as data collection using the Per ception 
Assessment Scale attitudinr 1 survey (Packard, R., et. al., 198&, March) and application of 
case-study procedures along with focus-group interviews (Packard & Dereshiwsky, 1989), 
to the preparation of summary documents and specific policy recommendations. Figure 1 , 
p. 6, depicts documentation procedures of data analysis, reduction and repoaing of related 
policy recommendations to the state legislature. A selected list of some of the research 
findings and summative reports may be found in the "reference" section of this paper. 

1 



The resultant assessment models and profiling procedures can be ..pplied to an\ 
school system, or for that maner, any other organization which has similarly idennfied the 
essential interrelated components for reaching functional goals. These models are depicted 
on pp. 7 through 10, as follows: (1) Figure 2. Interrelated Components of Program 
Support and Focus for Effecting Change and Reform in Education; (2) Figure 3 
Assessment Model for Projected District Readiness Levels; (3) Figures 4 and 5. District 
Readiness Profile of Strengths and Weaknesses. 
District System Readiness Level?; 
for Program Tmplementation 

Districts are extremely diverse in their ability to profit from the career ladder 
teacher-incentive intervention program. The discovery of individual districts' lack of 
readiness to support a major reform program, despite the existence of carefully formulated 
"uniform" legislative guidelines, has resulted in the development of an integrated model of 
support and focus factors which are essential for succe'^sful change effons (See Figure 2, 
p. 7). Each identifiable component within a system, which represents an organization's 
functional operations for meeting its specific goals, is important to consider when initiating 
maximally effective change or development of any one of its factors. All of the elements 
are interrelated and must be integrated when considering any essential ^art of the whole. 
With respect to successful reform, trying to change or improve one organizanonal elenieiii 
in isolation has not proven to be effective. 

Assessment & Profiling . The application of a variety of methodologic 
techniques has enabled the researchers to develop individual profiles derivea from 
assessment data. Figure 3, p. 8, shows a model which depicts the interrelated elencnts 
within district organizations which require assessment in order to determine the functional 
level at which each essential factor is operating. Figures 4 and 5 depict a sample profile 
derived from the assessment, thereby allowing the research evaluators to recommend 
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effects of divene readiness levels upon planning for improvement and success. 

Profiling has revealed that those factors originally assumed to be supportive of (he 
program goals of teacher and student progress have, instead, been detrimental in man> 
instances. For example, in some districts lacking readiness, early evidence suggests that 
factors such as evaluation systems, organizational climate, and administrator/distnct 
leadership and involvement have had a ne;5ative impact upon program implementation. 
Where little improvement is evident, teachers and students should not bear the burden of 
lack of program success alone. All possible areas of weakness which are "functional 
anchors" to reaching educational goals should be carefully considered with respect to 
identifying responsibility and accountability. In the past, there has been a tendency to 
blame teachers for lack of educational improvement without looking at other possible 
organizational ^^^ctors detrimental to success. 
Need for Planned I mprovement and Adequate 
Time for Sncce<!sful Change 

The diversity of readiness among districts, as well as differential readiness of 
essential operational elements within systems, results in the need for individual 
organizations to move forward at different rates. Also, individual diversities require a 
greater focus of attention on improvement of local system weaknesses, rather than on 
unrelated or "generic" inadequacies of other agencies. Explicitly acknowledging individual 
time and readiness needs is necessary in order to allow for integrated and successful 
change. 

The change process itself is a very important concept to integrate into any refonn 
program. The reasons for success or failure of past educational reform efforts have been 
carefully studied, in order to determine the implications for effective integration of ire 
career ladder concepts within each participating district. There needs to be ongoing 
monitoring of corresponding district indicators of operational effectiveness, such as an 




adccjuatcly dcvelopCu CuiuCUiUiu, opUiual cvaluatiun bybicrub, aud cfficieiu and accurate 
communications. 

The RoPation Rotwepn Teacher Pe rformance 
and Student Achievement 

A major area of research focus has been the connection between teacher 
performance and student achievement. Most of the pilot-test districts have not reached the 
highest level of their potential for objectively demonstrating the connection between teacher 
performance ev duation and student learning. In fact, the philosophical and technical 
diversity of districts is probably most evident in this panicular area of the study. For 
example, some districts continue to emphasize purely mechanical evaluation of teachers' 
methodology in the classroom rather than a thorough and careful identification of student 
outcomes resulting from teacher input variables. This practice erroneously continues to 
support the credibility of the assumption that teacher classroom activities accurately account 
for and/or predict student achievement. In order to be truly successful, a district must be 
able to validate its evaluation system based on actual gains in student achievement and not 
just on observational rating systems of classroom communication and activities. Some 
districts are beginning to make headway toward applying valid and reliable proced jres of 
curriculum application as related to tangible improvement in student outcomes. 

A recent finding in one of the pilot districts reveals that teachers should be evaluated 
based upon their actual performance levels, rather than solely on years of experience 'ind 
accumulated college credits, as has been somewhat naively assumed and applied in the 
past. This study is comprehensive, in Lhat it used three simultaneous measures of student 
achievement: ITBS reading, language and mathematics scores. Furthermore, it assessed 
the separate and interactive effects upon these measures of such potentially influential 
factors as teacher experience, education and grade level taught. 

Districts need to apply the latest quantitative techniques in order to develop their own 
valid and reliable measures of teacher and student performance. Several incennves withm 
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skills and assume leadership positions related to improvement of instructional pr jgrams 
Need for Program rn^fipiiafion 
and Refinement 

Directions for future rese'^rch include application of the proposed model of 
interrelated components of program support and focus factors for effective school refomi, 
in order to enable participating districts to monitor the current operating status of these 
factors with respect to program implementation. The opinions of government, 
business/industry and educational policy leaders are also being studied, in order to continue 
to involve outside agencies in the activities of the project and to apply the findings of policy 
research to decisions concerning the future status of t^ie career ladder program within the 
state. Program results have also been studied within the framework of the effective schools 
movement, along with related imnlications for etfects upon teacher improvement, 
administrative leadership, and improved accountability for student achievement. 

Conclusion 

In order for reform mo^'^ments to be mily successful, essential elements within 
organizations must be identified, assessed and profiled in order to apply appropriate 
intervention processes which effect change and improvement with respect to identified 
areas of weakness. This developmental model allows for individual diversities of readiness 
level ind sufficient time required for change within unique systems. 
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Figure 1 
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